Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cpython bdw nogc #1

Open
wants to merge 534 commits into
base: main
Choose a base branch
from
Open

Cpython bdw nogc #1

wants to merge 534 commits into from

Conversation

Yhg1s
Copy link
Owner

@Yhg1s Yhg1s commented May 6, 2019

Hackish replacement of refcounting with libgc mark-and-sweep.

(Probably) requires a libgc configured with --enable-redirect-malloc (that's what I'm using). Get libgc (BDW GC, aka Bohm GC) from https:////github.com/ivmai/bdwgc.

I configure with:
./configure --enable-sigrt-signals --enable-large-config --enable-mmap --enable-static --disable-shared --prefix=/some/path/to/bdwgc --enable-redirect-malloc

And then configure Python with:
CFLAGS="-I/some/path/to/bdwgc/include" ./configure --with-libgc="/some/path/to/bdwgc/lib/libgc.a /some/path/to//bdwgc/lib/libcord.a"

miss-islington and others added 30 commits July 21, 2019 12:44
(cherry picked from commit 8e3a738)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
…ythonGH-14870)

Repeat the command line arguments most recently entered before so the user can edit them.
(cherry picked from commit 35b87e6)

Co-authored-by: Ngalim Siregar <ngalim.siregar@gmail.com>
(cherry picked from commit 4214f1e)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit b530a44)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
…14872). (python#14900)

(cherry picked from commit 96e12d5)

Co-authored-by: Min ho Kim <minho42@gmail.com>
…s child with attach_mock (GH 14688) (pythonGH-14902)

* Clear name and parent of mock in autospecced objects used with attach_mock

* Add NEWS entry

* Fix reversed order of comparison

* Test child and standalone function calls

* Use a helper function extracting mock to avoid code duplication and refactor tests.
(cherry picked from commit 7397cda)

Co-authored-by: Xtreak <tir.karthi@gmail.com>
)

Reduce the number of unit tests run for the PGO generation task.  This
speeds up the task by a factor of about 15x.  Running the full unit test
suite is slow.  This change may result in a slightly less optimized build
since not as many code branches will be executed.  If you are willing to
wait for the much slower build, the old behavior can be restored using
'./configure [..] PROFILE_TASK="-m test --pgo-extended"'.  We make no
guarantees as to which PGO task set produces a faster build.  Users who
care should run their own relevant benchmarks as results can depend on
the environment, workload, and compiler tool chain.
(cherry picked from commit 4e16a4a)

Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
…honGH-14822)

(cherry picked from commit 1ebee37)

Co-authored-by: Tal Einat <taleinat@gmail.com>
(cherry picked from commit 7123ea0)

Co-authored-by: Tal Einat <taleinat@gmail.com>
)

Stop depending on tkinter import *.
(cherry picked from commit c6fd6c8)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
(cherry picked from commit f0807ab)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Verify that it appears to find roughly the right number of tests in the stdlib's testsuite.
(cherry picked from commit e95ac20)

Co-authored-by: Gregory P. Smith <greg@krypto.org>
(cherry picked from commit 2f224a0)


Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>

Automerge-Triggered-By: @brettcannon
…ree (pythonGH-14867) (pythonGH-14935)

* Fix the formatting in the documentation of the tostring() functions.

* bpo-34160: Document that the tostring() and tostringlist() functions also preserve the attribute order now.

* bpo-34160: Add an explanation of how users should deal with the attribute order.
(cherry picked from commit a3697db)

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
…nt/pi (pythonGH-14856) (pythonGH-14936)

* bpo-37399: Correctly attach tail text to the last element/comment/pi, even when comments or pis are discarded.
Also fixes the insertion of PIs when "insert_pis=True" is configured for a TreeBuilder.
…honGH-14934)

(cherry picked from commit 5380def)

Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
…ythonGH-14939)

(cherry picked from commit 123536f)

Co-authored-by: Steve Dower <steve.dower@python.org>
…e rather than listdir. (14942)

(cherry picked from commit 93e8aa6)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
…ads correctly (pythonGH-14593)

(cherry picked from commit 898318b)

Co-authored-by: Markus Mohrhard <markus.mohrhard@googlemail.com>
…le (pythonGH-14947)

Prior to this change the guard on an 'elif' used an assignment expression whose value was used in a later 'else' block, causing some confusion for people.

(Discussion on Twitter: https://twitter.com/brettsky/status/1153861041068994566.)

Automerge-Triggered-By: @brettcannon
(cherry picked from commit 544fa15)

Co-authored-by: Tzu-ping Chung <uranusjr@gmail.com>
…-14884)

Previously, pyc files in the embeddable distribution reported their
location as <build path>/<file stem>.py. This causes a little confusion
when interpreting stack traces as the file is in a (almost certainly)
incorrect location, and lacks the full relative path to Lib (e.g.
email/mime/image.py will only show image.py).

This change preserves the Lib relative location of the source file as a
path so that stack traces are (hopefully) less misleading and more
informative.

Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
(cherry picked from commit c4cda43)

Co-authored-by: Bill Collins <bilbocollins@gmail.com>
…ythonGH-14960)

(cherry picked from commit b1eb20e)

Co-authored-by: Steve Dower <steve.dower@python.org>
…nGH-14968)

(cherry picked from commit 91e4957)

Co-authored-by: Derek Keeler <d3r3kk@users.noreply.github.com>
…nGH-14959)

(cherry picked from commit 46ebd4a)

Co-authored-by: Tal Einat <taleinat@gmail.com>
The boxes for the font and highlight samples are now constrained by the overall config dialog size.  They gain scrollbars when the when a large font size makes the samples too large for the box.
(cherry picked from commit 3221a63)

Co-authored-by: Tal Einat <taleinat+github@gmail.com>
…ates (pythonGH-14975) (pythonGH-14984)

(cherry picked from commit 6b5f1b4)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
maxking and others added 23 commits September 4, 2019 18:20
…aders. (pythonGH-15239) (pythonGH-15686)

Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either:
- a case without trailing whitespace
- an invalid encoded word

https://bugs.python.org/issue37764

This fix should also be backported to 3.7 and 3.8

https://bugs.python.org/issue37764
(cherry picked from commit c5b242f)

Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com>
Modify the wheel event handler so it can also be used for module, path, and stack browsers.
Patch by George Zhang.
(cherry picked from commit 2cd9025)

Co-authored-by: GeeTransit <geetransit@gmail.com>
It should avoid dynamic lookup including `isinstance`.

This is a regression caused by pythonGH-5351.
(cherry picked from commit 8f9cc87)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
…ythonGH-15694)

(cherry picked from commit e4810b2)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
…pythonGH-15696)

(cherry picked from commit 9b51570)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
(cherry picked from commit 3038e87)

Co-authored-by: Andre Delfino <adelfino@gmail.com>
)

(cherry picked from commit d8c93aa)

Co-authored-by: Raymond Hettinger <rhettinger@users.noreply.github.com>
…honGH-15707)

As noted by @eryksun in [1] and [2], using _cleanup and _active(in
__del__) is not necessary on Windows, since:

> Unlike Unix, a process in Windows doesn't have to be waited on by
> its parent to avoid a zombie. Keeping the handle open will actually
> create a zombie until the next _cleanup() call, which may be never
> if Popen() isn't called again.

This patch simply defines `subprocess._active` as `None`, for which we already
have the proper logic in place in `subprocess.Popen.__del__`, that prevents it
from trying to append the process to the `_active`. This patch also defines
`subprocess._cleanup` as a noop for Windows.

[1] https://bugs.python.org/issue37380GH-msg346333
[2] https://bugs.python.org/issue36067GH-msg336262

Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
(cherry picked from commit 042821a)

Co-authored-by: Ruslan Kuprieiev <kupruser@gmail.com>
 Restart lines now always start with '=' and never end with ' ' and fill the width of the window unless that would require ending with ' ', which could be wrapped by itself and possible confusing the user.
(cherry picked from commit 38da805)

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
…thonGH-15479)

(cherry picked from commit 3ccdbc3)

Co-authored-by: Joannah Nanjekye <33177550+nanjekyejoannah@users.noreply.github.com>
…. (pythonGH-15728)

(cherry picked from commit 32a960f)

Co-authored-by: Greg Price <gnprice@gmail.com>
…H-15625)

RuntimeError is now raised in this case.
(cherry picked from commit 526a014)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
…calls (pythonGH-14462)

If FormatMessageW() is passed the FORMAT_MESSAGE_FROM_SYSTEM flag without FORMAT_MESSAGE_IGNORE_INSERTS, it will fail if there are insert sequences in the message definition.
(cherry picked from commit a656365)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
…onGH-15021) (python#15744)

The gdb manual[1] says the following for "document":

  The command commandname must already be defined.

[1] https://sourceware.org/gdb/current/onlinedocs/gdb/Define.html

And indeed when trying to use the gdbinit file with gdb 8.3, I get:

  .../cpython/Misc/gdbinit:17: Error in sourced command file:
  Undefined command: "pyo".  Try "help".

Fix this by moving all documentation blocks after the define blocks.

This was introduced in pythonGH-6384.
(cherry picked from commit 1f86fdc)

Authored-by: Florian Bruhin <me@the-compiler.org>
…onGH-13239)

* bpo-32587: Make winreg.REG_MULTI_SZ support PendingFileRenameOperations

* Address review comments.
(cherry picked from commit e223ba1)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
…15451)

There were about 14 files that are actually in the repo but that are
covered by the rules in .gitignore.

Git itself takes no notice of what .gitignore says about files that
it's already tracking... but the discrepancy can be confusing to a
human that adds a new file unexpectedly covered by these rules, as
well as to non-Git software that looks at .gitignore but doesn't
implement this wrinkle in its semantics.  (E.g., `rg`.)

Several of these are from rules that apply more broadly than
intended: for example, `Makefile` applies to `Doc/Makefile` and
`Tools/freeze/test/Makefile`, whereas `/Makefile` means only the
`Makefile` at the repo's root.

And the `Modules/Setup` rule simply wasn't updated after 961d54c.

https://bugs.python.org/issue37936
(cherry picked from commit 5e5e951)

Co-authored-by: Greg Price <gnprice@gmail.com>
…H-15623)

winerror_to_errno() is no longer automatically generated.
Do not rely on the old _dosmapperr() function.
Add ERROR_NO_UNICODE_TRANSLATION (1113) -> EILSEQ.
(cherry picked from commit 19052a1)

Co-authored-by: Zackery Spytz <zspytz@gmail.com>
…-12251)

Authored-By: blueyed <github@thequod.de>
(cherry picked from commit 8d64bfa)

Co-authored-by: Daniel Hahler <github@thequod.de>
(cherry picked from commit 63c98ed)


Co-authored-by: Julien Palard <julien@palard.fr>

Automerge-Triggered-By: @JulienPalard
…error messages (pythonGH-14310)

(cherry picked from commit 9d60706)

Co-authored-by: Xtreak <tir.karthi@gmail.com>
@larryhastings
Copy link

larryhastings commented Sep 10, 2019

Here's a shell script that successfully builds the whole shebang, following build instructions from both Yhg1s and ivmai:

mkdir libgc
cd libgc
dir=`pwd`
git clone -b cpython-bdw-nogc https://github.com/Yhg1s/cpython.git
git clone https://github.com/ivmai/bdwgc.git
cd bdwgc
git clone git://github.com/ivmai/libatomic_ops.git
./autogen.sh
./configure --enable-sigrt-signals --enable-large-config --enable-mmap --enable-static --disable-shared --prefix="$dir" --enable-redirect-malloc
make -j
make check
make install
cd ..
cd cpython
CFLAGS="-I$dir/include" ./configure --with-libgc="$dir/lib/libgc.a $dir/lib/libcord.a"
make -j

It will create a "libgc" directory then do all its work in there. I think it may not work properly if there is a space anywhere in the absolute current working directory.

Yhg1s pushed a commit that referenced this pull request Mar 1, 2021
* bpo-40791: Make compare_digest more constant-time.

The existing volatile `left`/`right` pointers guarantee that the reads will all occur, but does not guarantee that they will be _used_. So a compiler can still short-circuit the loop, saving e.g. the overhead of doing the xors and especially the overhead of the data dependency between `result` and the reads. That would change performance depending on where the first unequal byte occurs. This change removes that optimization.

(This is change #1 from https://bugs.python.org/issue40791 .)
Yhg1s pushed a commit that referenced this pull request Jun 6, 2023
…ythonGH-23767)

The existing volatile `left`/`right` pointers guarantee that the reads will all occur, but does not guarantee that they will be _used_. So a compiler can still short-circuit the loop, saving e.g. the overhead of doing the xors and especially the overhead of the data dependency between `result` and the reads. That would change performance depending on where the first unequal byte occurs. This change removes that optimization.

(This is change GH-1 from https://bugs.python.org/issue40791 .)
(cherry picked from commit 3172936)

Co-authored-by: Devin Jeanpierre <jeanpierreda@google.com>
Yhg1s pushed a commit that referenced this pull request Jun 6, 2023
* bpo-40791: Make compare_digest more constant-time.

The existing volatile `left`/`right` pointers guarantee that the reads will all occur, but does not guarantee that they will be _used_. So a compiler can still short-circuit the loop, saving e.g. the overhead of doing the xors and especially the overhead of the data dependency between `result` and the reads. That would change performance depending on where the first unequal byte occurs. This change removes that optimization.

(This is change GH-1 from https://bugs.python.org/issue40791 .)
(cherry picked from commit 3172936)

Co-authored-by: Devin Jeanpierre <jeanpierreda@google.com>
Yhg1s pushed a commit that referenced this pull request Jun 6, 2023
The existing volatile `left`/`right` pointers guarantee that the reads will all occur, but does not guarantee that they will be _used_. So a compiler can still short-circuit the loop, saving e.g. the overhead of doing the xors and especially the overhead of the data dependency between `result` and the reads. That would change performance depending on where the first unequal byte occurs. This change removes that optimization.

(This is change GH-1 from https://bugs.python.org/issue40791 .)
(cherry picked from commit 3172936)

Co-authored-by: Devin Jeanpierre <jeanpierreda@google.com>
Yhg1s pushed a commit that referenced this pull request Oct 13, 2023
…es (#1… (python#108688)

This reverts commit 08447b5.

Revert also _ctypes.c changes of the PyDict_ContainsString() change,
commit 6726626.
Yhg1s pushed a commit that referenced this pull request Jun 5, 2024
…ython#119498) (#1… (python#119905)

Revert "[3.12] pythongh-69214: Fix fcntl.ioctl() request type (python#119498) (python#119505)"

This reverts commit 078da88.

The change modified how negative values, like termios.TIOCSWINSZ, was
treated and is actually backward incompatible.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.